Naïve v. expert intuitions: An empirical study of acceptability judgments

Judgments about the grammaticality/acceptability of sentences are the most widely used data source in the syntactic literature. Typically, syntacticians rely on their own judgments, or those of a small number of colleagues. Although a number of researchers have argued that this is problematic, there is little research which systematically compares professional linguists’ intuitions with those of linguistically naïve speakers. This paper examines linguists’ and nonlinguists’ judgments about one particular structure: questions with long distance dependencies. Linguists’ judgments are shown to diverge from those of nonlinguists. These differences could be due to theoretical commitments (the conviction that linguistic processes apply ‘across the board’, and hence all sentences with the same syntactic structure should be equally grammatical) or to differences in exposure (the constructed examples of this structure found in the syntactic literature are very unrepresentative of ordinary usage). Whichever of these explanations turns out to be correct, it is clear that linguists’ judgments are not representative of the population as a whole, and hence syntacticians should not rely on their own intuitions when testing their theories.